The invention relates generally to testing of software, and more specifically to systems and methods for automated testing of software that executes business processes.
Business process application software allows for the definition and execution of business processes in a computer system. Examples of business processes include updating customer information, receiving and fulfilling orders, synchronizing customer information stored on multiple computer systems, and generating a price quote. Business processes are often associated with data descriptions and transformation rules for accessing individual data items, such as a customer name or a unit price. A business process specifies a sequence of activities to be performed in a specified order, and may specify conditional and repeated execution of activities. Business process application software can execute a business process, prompting for or retrieving input data as needed, and produce results or effects, such as output data or execution of database transactions. A business process application configuration includes a set of business processes and associated data, including data descriptions and transformation descriptions, which specify how to execute one or more particular business processes using general-purpose business process application software, such as SAP™ or Siebel™. Configuration information is typically represented as data stored in disk files or in an online repository, depending on the particular business process application software. The business process software can load the configuration information, including the business process, and subsequently execute the business processes. For example, a shipping company may have a business process application configuration including a computer system running SAP™ software with a ShipmentReceived business process, which is to be executed when a shipping container arrives at a transit hub. The ShipmentReceived business process updates an inventory database of shipping container locations and invokes another computer system to route the shipping container to its destination. The shipping company may have a second business process application configuration consisting of another computer system running Siebel software with a ConfirmOrder business process, which is to be executed when an order has been processed. The ConfirmOrder business process, which is invoked by an order processing system, retrieves the customer's e-mail address from Siebel and sends an e-mail notice to the customer describing the order.
It is desirable to be able to automatically test the execution of a business process application configuration by a business process application for correctness. Testing a software program involves interacting with the program in a test environment and verifying that the program operates correctly, based on the program's behavior in response to input supplied during the interaction. The interaction may be manual testing performed by a human or automated testing performed by a test program. Automated testing is desirable because human effort is not necessary to perform the testing, so automated testing can be less expensive and more consistent than manual testing, once the tests have been developed. However, substantial time and effort may still be involved in creating the tests, which are typically computer programs, that will perform the automated testing. Test automation systems, which are typically computer programs, have been developed to reduce the time and effort involved in creating automated tests.
Co-pending U.S. patent application Ser. No. 11/251,281, System And Method For Testing Business Process Configurations, B. Larab, et al., discloses systems and methods for testing business process configurations, in which the testing is based upon complex test elements, which interact with specific screens or transactions of a specific application, e.g., SAP™ or Siebel™, to provide input data and read output data from the application. A complex test element includes a parameter list, which describes the names and types of input and output parameters that are expected and produced, respectively, by the transaction, and a list of elements to be invoked to perform the transaction. The complex test element passes values received from the user for the input parameters to the transaction when the test is executed, and passes values produced by the transaction back to the testing system so that the user can verify that the test's output is correct, or perform an action based upon the test's output. The test elements can be used as building blocks to create automated tests. For example, a user may select a test element associated with a specific Purchase Order transaction, add the element to a test, and provide input data for the element. The test automation system can then execute a test of the Purchase Order transaction using the element and input data. Multiple test elements can be strung together in a sequence to create tests of multiple features, e.g., multiple transactions, and test elements that perform conditional branching can be used to create tests that execute certain elements conditionally or repeatedly. An element that performs an application-specific action, such as a transaction, is created by combining simpler elements that perform actions such as setting or getting a value of a text field or selection box in an application user interface. For example, an element that performs a Purchase Order transaction could be created by combining elements that set the values of each field of the Purchase Order transaction in a user interface. If the Purchase Order transaction includes input fields for an item name and quantity, the Purchase Order element would be able to indicate to a user of the testing system that an item name and a quantity are expected as input parameters to the transaction. The Purchase Order element would also be able to invoke field setting elements that set the item name and quantity input fields to parameter values provided by a user of the test automation system.
Test elements can be created manually by a user interacting with the testing system. To create a new test element for a transaction, the user first perceives the format of the input data, i.e., the names and types of the input fields of the transaction. Then the user may add field setting elements to the new test element to set values of the transaction's input fields. The user may also add field getting elements to retrieve output values produced by the transaction. The test element can then be used by the testing system to invoke the transaction. For example, a user could create a Purchase Order test element by first looking at the Purchase Order transaction screen or documentation to determine that the purchase order transaction's input includes an item name and a quantity. The user would then create an empty complex element and add field setting elements for the item name and quantity fields to the empty complex element. The Purchase Order test element could then be included in business process tests to invoke the Purchase Order transaction.
This manual process of creating test elements can be resource-intensive and error-prone because it depends on a human user. A human user may make mistakes or omissions in creating the test elements, and the time taken by a human user to create test elements for an application may be substantial and costly. An application may have hundreds or thousands of possible transaction screens or types, in which case the expense and possibility of error may be substantial.
One way to automate the test creation process is to introduce a recording feature, in which the test system records the user's actions when a transaction screen is filled in. However, this approach sometimes can lead to difficulties, such as tests being overly dependent upon details of the application screens or the version of the application, as described in the above-referenced patent application Ser. No. 11/251,281.
Furthermore, the manual process of creating a test element generally produces test elements that are dependent on the particular language version (e.g., English or Japanese) of the application, because the descriptive text associated with each field on the application screen is used to identify the field in the test elements. If the descriptive text changes, for example, as a result of switching from an English-language version of the application to a Japanese-language version, then the test element will become inoperable. The descriptive text may also change for other reasons, e.g., if the application vendor changes the text in a new version of the application.
Because of the resource-intensive, error-prone, and language-dependent nature of the manual process for generating test elements, it would be desirable to be able to automatically generate test elements.